Add workflow to trigger OCP tests #12542
Conversation
|
Start a new ephemeral environment with changes proposed in this pull request: ocp4 (from CTF) Environment (using Fedora as testing environment) Fedora Testing Environment Oracle Linux 8 Environment |
yuumasato
force-pushed
the
add_workflow_to_trigger_ocp_tests
branch
6 times, most recently
from
91760f6
to
f677259
Compare
|
🤖 Trigger prow tests based on changed rules /test 4.17-e2e-aws-ocp4-moderate-rev-4 /test 4.17-e2e-aws-ocp4-pci-dss-node |
yuumasato
force-pushed
the
add_workflow_to_trigger_ocp_tests
branch
3 times, most recently
from
5aa9f46
to
2a785c1
Compare
|
🤖 Trigger prow tests based on changed rules /test 4.17-e2e-aws-ocp4-pci-dss-3-2 Note: if a test is not started it can be that a CI Job is not configure for that particular profile or product. Click here to see all the relevant profiles |
|
@github-actions[bot]: The specified target(s) for
Use
In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
yuumasato
force-pushed
the
add_workflow_to_trigger_ocp_tests
branch
3 times, most recently
from
695fc80
to
4934c26
Compare
Use CTF to identify changes in OCP rules, identify the profiles selecting these rules, and start Prow tests.
yuumasato
force-pushed
the
add_workflow_to_trigger_ocp_tests
branch
from
4934c26
to
84b6e16
Compare
Not all profiles are configured in Prow. This uses a hardcoded list of profiles to filter out the "untestable" ones.
yuumasato
force-pushed
the
add_workflow_to_trigger_ocp_tests
branch
from
84b6e16
to
b7c47b0
Compare
|
🤖 Trigger prow tests based on changed rules /test 4.17-e2e-aws-ocp4-high-node Note: if a test is not started it could be that a CI Job is not configure for that particular profile or product. Click here to see all the relevant profiles
|
yuumasato
force-pushed
the
add_workflow_to_trigger_ocp_tests
branch
from
b7c47b0
to
0cf1bbf
Compare
|
I have dropped the artificial change: b7c47b0. |
| if: ${{ steps.ctf.outputs.CTF_OUTPUT_SIZE != '0' && (contains(steps.product.outputs.prop, 'ocp4') || contains(steps.product.outputs.prop, 'rhcos4')) }} | ||
| id: profiles_to_test | ||
| run: | | ||
| OCP_VERSIONS=(4.17 4.16) |
There was a problem hiding this comment.
We'll need to manage this in addition to any OCP version changes in openshift/releases?
There was a problem hiding this comment.
Yes.
Another approach is to trigger the version-less tests, like e2e-aws-ocp4-high-node.
Sidenote: currently the version-less tests are running on 4.16, not the latest 4.17, 🙈
I considered running on two OCP versions, just to have more default coverage.
But it can be wasteful. I'm also okay with running only the version-less tests.
| for rule in $RULES; do | ||
| readarray -t TEMP <<< $(grep -lr -e "- ${rule}\$" build/*/profiles | sort) | ||
|
|
||
| # Let's ilter out profiles for which we don't have a CI job configured |
|
|
||
| # Let's ilter out profiles for which we don't have a CI job configured | ||
| # Here is an example of how to quicly update this variable in the future | ||
| # TESTED_PROFILES=$(grep -r PROFILE= ./ComplianceAsCode-content-master__4.16.yaml | sort -u | sed 's/.*export PROFILE=\(.*\)/\1/') |
There was a problem hiding this comment.
This must be run from the openshift/releases repository, right?
There was a problem hiding this comment.
I don't have a good work around for this.
We could clone the repo and parse the files, or issue a /test comment and parse the comment from the ci bot.
I added it because we don't have tests for all profile ids.
You can see in previous comments that pci-dss-3-2 was chosen for tests, but we don't have CI wired up for that.
The version-less jobs are expected to be running on latest OCP.
Let's directly parse the source of CI Job configuration to know what profiles we can trigger jobs for. Also, move it out of the rule iteration loop. We just need to define it once.
These changes should be identified by CTF and Prow tests should be started automatically.
|
🤖 Trigger prow tests based on changed rules /test e2e-aws-ocp4-high-node Note: if a test is not started it could be that a CI Job is not configure for that particular profile or product. Click here to see all the relevant profiles
|
|
Code Climate has analyzed commit 7b46c13 and detected 0 issues on this pull request. The test coverage on the diff in this pull request is 100.0% (50% is the threshold). This pull request will bring the total coverage in the repository to 60.9% (0.0% change). View more on Code Climate. |
|
@yuumasato: The following tests failed, say
Full PR test history. Your PR dashboard. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
Description:
OCP4orRHCOS4profile that selects it to be tested.Rationale:
Review Hints: